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family support and parenting programs, based on the field's history of 
experience, that can be used as the basis for future decisions about programs 
to improve parenting support or support of good parenting practices. The 
recommendations are: (1) develop services that focus on parent-child 

interactions; (2) tailor services to meet the needs of unique regional, 
cultural, and ethnic groups; (3) start intervention early, and acknowledge 
the potential need for a continuity of services; (4) have reasonable 
expectations about the time commitment required of families and providers; 

(5) recognize that factors in parents' social and cultural context have an 
impact on parenting; (6) work with other providers to form a system of 
efficient and comprehensive services; (7) use evaluation strategies that 
reflect and support the field's move toward more comprehensive initiatives; 

(8) examine child and parent outcomes and needs longitudinally; (9) choose 
measures that reflect intended program outcomes; (10) examine the relation 
between parent and child outcomes; (11) establish mutually beneficial 
relationships between evaluators, providers, and child development 
researchers; and (12) consider using cost-effectiveness analyses as a method 
for measuring and reporting program results. Contains 36 references. (TJQ) 
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Preparing the Next Generation: 

Recommendations for Meeting the Accountability Demands 
on Family Support and Parenting Programs 

In this era of increased accountability, family support and parenting programs are under 
pressure to demonstrate their results. To increase their chances of success and survival, 
programs must look to and build upon the lessons from the past three decades of research and 
evaluation. Their results will determine their future. 

Increased program accountability has implications for several groups associated with 
family support and parenting programs, including policymakers, providers, and evaluators. The 
responsibility for producing programs that can prove their worth extends beyond program 
designers and implementers, to those associated with the programs at any stage of their 
development and implementation. Policymakers are accountable to their constituents for 
identifying and funding worthwhile programs. They are also responsible for examining the 
current system of services to determine any changes that can produce better overall results. 
Providers are accountable to program participants and funders for proving their success. They 
must implement programs that measure and meet intended outcomes. Evaluators are responsible 
for building the capacity of programs to measure these outcomes and provide information that 
informs decisions and can be used to continuously improve services. Finally, it is important to 
remember that all groups are accountable to the children whom family support and parenting 
programs ultimately serve. The scope of responsibility has thus extended so that each group is 
accountable for successfully improving the outcomes of children and their families. 

As policymakers, providers, and evaluators gear up to meet the challenges of 
accountability, reflection on what is now known about family support and parenting programs 
from its rich history of research and evaluation is necessary to determine the best steps for the 
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future. Such reflection should include an assessment of what it means to be accountable and what 
can be learned about services and evaluation to successfully meet the challenges of 
accountability. 

This paper addresses both of these issues. It identifies 12 recommendations for the future 
of family support and parenting programs that are based on the field’s history of experience and 
that can be used as the basis for future decisions about programs that aim to improve parenting 
or support good parenting practices. 

Meeting the Demands of Accountability 

Family support and parenting programs exist for parents who need help in acquiring the 
support, resources, and skills that encourage positive child development. They most often serve 
families in low-income areas, where poverty threatens the development of children and the 
ability of parents to protect, support, and nurture their children. In addition, programs that are 
universal in focus offer services to any parents who desire them. 

The pathways family support and parenting programs should take to meet their intended 
outcomes are less obvious than the need for such programs. To prepare the next generation of 
programs to meet higher expectations for successful results, we generated 12 recommendations 
that examine the lessons learned from the past and use them in forming practices for the future. 
The recommendations offer advice on how to design and evaluate programs to make them more 
successful at measuring and meeting the demands of accountability. 
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Recommendations in Brief 



1 Develop services that focus on parent-child interactions. Programs should emphasize 
quality interactions between parents and children because they lead to better developmental 
outcomes for children. It is important not only to serve parents and children together and to 
provide a time for parents and children to interact but also to focus on improving the 
dynamics of the interaction that occurs. Program curricula should include provisions for 
developing the types of interactions and ways of promoting behavior that existing research 
shows are effective in improving child outcomes. 

2. Tailor services to meet the needs of unique regional, cultural, and ethnic groups. In a 

nation of diverse communities with equally diverse values, needs, and ideas about results, 
programs should match their designs to the unique needs of their communities without 
sacrificing important core attributes of interventions such as intensity and duration of 
services. This means including provisions to assess distinct needs and involving community 
members in the design, implementation, and evaluation of services. 

3 . Start interventions early, and acknowledge the potential need for a continuity of 
services. The challenges of parenting begin during pregnancy, and intensify immediately 
following birth. As it is easier to prevent the negative effects of poor parenting practices than 
to reverse them, this recommendation calls for early intervention. The availability of 
services or supports into the child’s later years are also important. Continuity can take a 
variety of forms, such as appropriate services in a child’s later years, or coordination with 
other programs such as Head Start, after school programs, or parent involvement programs 
when the child enters school. It can also mean developing the capacity of parents to maintain 
a good relationship with their children and ensuring that parents have access to supports and 
resources when needed. 

4. Have reasonable expectations about the time commitment required of families and 
providers. The employment and economic requirements that resulted from welfare reform 
leave many families with a limited amount of time during the day to spend in programs. In 
addition, the need to cut costs and serve families on a restricted budget requires that 
providers have only a limited time to spend in direct services to families. This 
recommendation calls for programs to carefully develop the frequency and quality of time 
families and providers spend in direct service contact. It calls for programs to become more 
efficient without sacrificing quality. It may mean, for example, that programs enhance the 
capacity of participants to develop networks of support in their communities that can be used 
when providers are not available or families do not have large chunks of time to devote to 
direct services. Programs may have to embed their services within others that families 
receive (i.e., child care). It may also mean that programs focus on sustaining the ability of 
families to continue to be good parents and obtain support when direct services end. 

5. Recognize that factors in parents’ social and cultural context have an impact on 
parenting. Family support and parenting programs recognize that several variables present 
in parents’ environments such as poverty, unemployment, undereducation, or a lack of social 
supports have an impact on their ability to be good parents. While it may be outside of their 
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direct purview to address each of these variables, programs must use tactics such as 
collaboration or coordination with other service systems to mitigate the negative impacts of 
these problems on parenting. 

6 . Work with other providers to form a system of efficient and comprehensive services. 

This recommendation follows directly from the one above, in that families need access to a 
variety of services in order to sustain their ability to be good parents. Family support and 
parenting programs alone cannot provide each of these services, however they must work 
with other providers to develop pathways of service access for families. This 
recommendation asks programs to consider where their services fit within the overall system 
of services in their communities, and to consider how they can work with these services to 
meet broad family needs. This may also translate into developing a system of shared 
referrals and tracking of referrals. 

7. Use evaluation strategies that reflect and support the field’s move toward more 
comprehensive initiatives. As programs begin to incorporate the provisions needed to meet 
the multiple and changing needs of the families they serve, they need better information 
about the success of such comprehensive strategies. This may mean collaborating with other 
local service providers to track families across services, or developing the capacity within 
programs to track information that is useful to improving their own services and outcomes. It 
may also mean that many programs are ready to move beyond a traditional reliance on 
experimental and quasi-experimental designs common to research and development efforts 
toward more cost-effective and alternative forms of evaluation. 

8. Examine child and parent outcomes and needs longitudinally. Although programs may 
serve children and parents for only a specific period of time, it is important that they build the 
capacity to continue measuring program results over time. While the obvious benefit of such 
examination is in capturing the program’s long-term success, it is equally beneficial to use 
such an opportunity to periodically assess the long-term needs of parents and children. In 
this way, data collection over time is not only beneficial for purposes of evaluation, but is 
beneficial for those who are tracked. When designing an evaluation that incorporates a 
longitudinal focus, it is important to include short-term indicators as well. 

9. Choose measures that reflect intended program outcomes. As the need to measure 
outcomes intensifies for programs, they must be confident that their measures adequately 
reflect their progress in achieving intended results and are applicable to the populations they 
serve. This means that programs should carefully choose instruments or indicators that will 
be meaningful once obtained. This also means reassessing the usefulness of traditional 
measures and using measures or indicators that are equally reliable and useful, but perhaps 
easier and less costly to use. 

10. Examine the relation between parent and child outcomes. Programs that carefully track 
child and program outcomes should include provisions for examining the relationship 
between changes in parent outcomes and changes in child outcomes. Parent-child interaction 
itself should also be examined. The theory underlying any program that attempts to improve 
or support positive parenting holds that improvements in parent outcomes are linked to better 
child outcomes. It is thus not enough to show that either one or both types of outcomes have 
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changed, but to measure the link between the two that is implicit in the programs’ theories. 



11. Establish mutually beneficial relationships between evaluators, providers, and child 
development researchers. The gradual expansion of family support and parenting programs 
beyond research and development efforts has also meant a gradual shift from experimental 
and quasi-experi mental evaluation designs to alternative forms of evaluation such as 
participatory evaluation and process and implementation studies. These newer forms of 
evaluation require that evaluators work closely with providers in mutually-beneficial 
relationships that are well-suited to the efficiency and measurement demands of 
accountability while maintaining some level of objectivity. This recommendation calls for a 
continuation of such collaboration, as well as an expansion of this practice. It encourages 
evaluators to help programs develop the capacity for self-evaluation and to assist programs in 
translating outcome information into a useable format for policymakers. 

12 Consider using cost-effectiveness analyses as a method for measuring and reporting 
program results. Family support and parenting programs should consider conducting cost- 
effectiveness studies as part of meeting the demand for accountability. These analyses 
determine the effectiveness of a program in achieving results in relation to how much a 
program costs. They determine how much cost is required to achieve a desired level of 
desired change. While the procedures used to conduct these analyses are often technical and 
complex, they provide valuable decision making criteria for policymakers and providers. 



Methodology 

Using a relational database, the Harvard Family Research Project (HFRP) compiles 
detailed information about preventive, family-oriented programs serving children and youth and 
their evaluations. The database currently includes complete information on 30 programs (see 
Appendix A for a list of the programs). All programs must satisfy two criteria before they are 
included in the database First, they must meet the definition of a family support program. 
Specifically, the programs must focus, at least in part, on prevention and target an entire 
community or at-risk groups in communities. Secondly, the programs must have evaluations 
with either an experimental or quasi-experi mental design. All evaluations in the database have 
either a control group that received no intervention, a comparison group that is matched with the 
intervention group on important characteristics, or another form of comparison group. 

The database stores information that allows for systematic reviews of the evaluation 
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literature. The evaluations are coded in a way that allows many program and methodological 
variables to be summarized and used in cross-study comparisons without losing the important 
nuances of the studies. 

The literature on family support and parenting programs, evaluation, and child 
development was used to identify key areas for analysis. Issues identified by multiple authors as 
important for decision making about the direction in which the field should move were sought. 
Many of these areas are reflected in the recommendations. 

Coding for the evaluations allowed for frequency counts of the important dimensions 
identified in the literature scan. Factors relating to program focus, services, and design in 
addition to evaluation issues were categorized based on the coding. Once frequency counts were 
completed, simple statistical analyses examined differences among the program. The 
recommendations that emerged indicate areas where a preponderance, or an overwhelming lack, 
of program evidence was present. 




Harvard Family Research Project 



6 



Recommendations and their Rationale 



1. Develop services that focus on parent-child interactions. 

Many years of child development research suggest that parent-child 1 interactions can 
contribute to positive child outcomes in most areas of child development. Training should 
prepare providers to intervene around specifics of the parent-child relationship, and to influence 
attitudes and knowledge about the basics of parenting (i.e. discipline, promoting literacy). To 
make the most of their children’s developmental potential, especially in the early years, parents 
should be able to understand their children’s signals and intentions (sensitivity ) 2 and respond 
appropriately (responsivity). They should also coordinate their interactions with children based 
on the children’s signals and responses (synchrony). In addition, parents should be nurturing, 
respond to their children’s emotions and be appropriately expressive of their own (emotional 
availability), and foster a close emotional bond with their children (attachment) 3 . Parents who 
behave in these ways have children who have well-developed social skills and peer relations, 
exhibit good intellectual development including language development, play in age-appropriate 
ways, express their emotions appropriately, and have a positive sense-of-self . 4 

Because children have an effect on the parenting they receive, parents need to be able to 
adapt their behaviors to the characteristics of their children. For example, children with different 
personality characteristics and ways of responding to the world (temperament) may require 
different methods of parenting 5 . Parents who are sensitive and responsive to their infant’s inborn 
tolerance for stimulation can provide the appropriate levels needed for good development, and 
can avoid the potential negative consequences of overstimulation. Although it is difficult to give 
a single prescription for good parenting given this need to acknowledge each child’s inborn 
tendencies, fostering liberal doses of sensitivity, flexibility, and communication on the part of 
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parents is a promising practice 6 . “[A]ny program giving prescriptions about ‘the right way to do 
it’ will clearly be deficient if it does not also direct parents’ attention to individuality and to the 
need to be flexible in the approach to childrearing .” 7 

This advice from the child development literature has several implications for family 
support program design. It is important not only to serve parents and children together and to 
provide a time for parents and children to interact, but also to focus on improving the kind, style, 
and intensity of interaction that occurs. Providers should focus the intervention so it, at least in 
part, provides feedback on parent-child interaction. Providers can model appropriate behavior, 
critique parents’ interactions, reinforce positive behaviors, or even tell parents about more 
appropriate or adaptive ways of behaving once a relationship between the provider and the 
family has been established. They can alert parents to the opportune time to talk with their 
children, such as while diapering or feeding, and point out the importance of language and 
communication. Programs should thus include provisions for developing the types of 
interactions that existing research shows are effective in improving child outcomes. In 
particular, programs should include a focus on the aspects of parenting that are proven to be 
important to child development. The most important lesson to be learned here is that programs 
can influence the actual behavior of parents toward their children with the goal of having 
positive interaction occur more often than not. While many programs focus on improving 
parents’ attitudes, beliefs, perceptions, knowledge, and expectations about childrearing and child 
development, many do not include the same emphasis on improving parent-child interactions. 
This finding is significant, given the fact that developmental research has not yet established the 
same strong empirical link between improvement in knowledge, attitudes, and beliefs and 
children’s actual development, as it has between specific aspects of parent-child interactions and 
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child development. 

Evidence from Program Experience — The programs used for developing our 
recommendations focused on the details of parent-child interactions, showed positive interaction 
and child development outcomes. For example, the Houston Parent-Child Development Center 
(PCDC) videotaped mothers and children during play groups and had one-on-one or group 
discussions with mothers about the quality of their interactions. The PCDC’s evaluation 
revealed strong positive parent-child interaction outcomes at the end of the intervention Parents 
who participated in the program were more likely to show positive behavior toward their 
children in a teaching situation. They were more affectionate, used less criticism, and were more 
encouraging of their children’s verbalizations than were the mothers who did not participate. 

The Birmingham PCDC focused on parent-child interaction by having mothers care for 
their own children in a supervised nursery setting. Mothers received guidance from teaching 
mothers about positive parent-child interactions. Birmingham program mothers differed from 
mothers who did not participate in a waiting room situation and a teaching situation. Program 
mothers were more positive in both situations than were comparison mothers. 

Programs can benefit from these examples and incorporate a focus on the quality of the 
parent-child relationship Home visits, which are a common feature of family support programs, 
provide an ideal opportunity for parent-child interaction instruction and discussion. For 
example, providers can observe interaction during the visits, and then focus on positive parent- 
child interaction through reflection about parenting or gently directive comments. 

2. Tailor services to meet the needs of separate regional, cultural, and ethnic groups. 

Different regional, cultural, and ethnic groups face unique challenges, and there may not 
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be one single intervention strategy that works for all groups 8 . As a result, programs should take 
time to assess the distinct needs of the groups they serve, and involve community members in the 
design, implementation, modification, and evaluation of services. This approach will ensure they 
deliver services that are appropriate and needed and which are in line with existing value 
structures and beliefs. 

This recommendation is particularly useful for programs that attempt to replicate 
programs that were originally designed for different ethnicities and communities. In designing 
parenting programs for different cultural groups, programs should note possible differences in 
the cultural goals of childrearing 9 . For example, white middle class American culture may 
emphasize independence in children, while other cultures may emphasize the attributes of 
cooperation and social relatedness 10 . It is also important to investigate which concerns are most 
important for different groups as well as the preferred ways of receiving services. For example, 
some groups may prefer center-based services to home visiting. 

Evidence from Program Experience — Evidence that supports the need to design 
programs that meet the needs of groups from different regions was present for several programs 
we examined. For example, while the Mother-Child Home Program (MCHP) was a successful 
intervention for low-income families in the United States, the program’s replication in Bermuda 
was less successful at influencing child cognitive development through mothers’ verbal 
interaction during play. Program providers and evaluators attributed this finding to the fact that 
the program had little to improve upon in this domain. They determined, in retrospect, that most 
children in Bermuda already functioned well in this domain because they participated in group 
child care that was similar to the MCHP program in focus. 

Evidence from program experience also indicates that materials used in one program may 
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not be appropriate for groups in other locations. For example, the Florida First Start program 
(not summarized in the profiles) is an example of such a replication problem. This program, 
implemented throughout the state of Florida, was modeled after the Missouri Parents as Teachers 
program (see profile for New Parents as Teachers), a program designed to serve all new parents. 
Florida First Start, however, served low-income families, most of whom had less than a high 
school education. The Parents as Teachers materials given to parents in Florida were written at a 
level that most parents did not understand and thus could not apply. As a result, families did not 
benefit as much from the programs as did parents in Missouri. 

It is important to be aware of program participants’ abilities when deciding how to 
present the goals of a program as well. A program teaching parents how to do cognitive tasks 
with their children encountered a problem because of how literally the teenage mothers in the 
program took the instructions 1 1 . When showing the mothers how to determine if their child had 
developed the knowledge that objects still exist when they are out of sight, the providers used the 
example of covering an object with a tablecloth. One teen mother replied, “But we ain’t got no 
tablecloth.” 

The Haitian Perinatal Intervention Project (a Child Survival/Fair Start project) is another 
example of the need to be sensitive to the child rearing beliefs of different groups. This program 
attempted to foster infant learning through maternal interaction. The program did not 
acknowledge, however, that Haitian mothers did not accept that their infants could learn when 
they were less than one-year-old. As a result, the Haitian program was not successful in 
achieving its goal. Mothers in the program did not have a true interest in changing interactions 
with their infants, because their beliefs were not in line with the explicit goal of the program 12 . 

When choosing program components, programs must carefully choose those that are 
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appropriate for and desired by the group being served. In cases of replication, the program may 
need to redesign specific service elements to meet the needs of different groups. For example, 
the three Parent Child Development Centers (in Houston, Birmingham, and New Orleans) were 
likely effective at improving parenting because each project aligned specific program elements 
with the culture of each location (the Houston program served families in the barrio (100% 
Latino), and the New Orleans program served only African-American families). All of the 
programs hoped to ensure the same set of long-term outcomes by enrolling families soon after 
the child’s birth and by focusing intense services on the mother until the child was three. The 
Houston program surveyed the community and designed services based on the results of the 
survey. The program provided home-based services for the first year and involved other 
members of the family in services. It also provided some optional bilingual education since most 
parents identified this as a need. The New Orleans program served parents in the center and 
spent time focusing on the mother as an adult as well as a parent. These programs provide an 
example of an effective way of reproducing a program model with minor, yet important, 
revisions. 

3. Start interventions early, and acknowledge the potential need for a continuity of 
services. 

The challenges of parenting begin during pregnancy, and become intense immediately 
following birth. As it is easier to prevent the negative effects of poor parenting practices than to 
reverse them, this recommendation calls for early intervention. Parents may more readily accept 
interventions if they are presented early on and, thus, are not seen as a judgment about their 
ability as parents. The first three years of life have long been considered a critical period for 
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development. Recent evidence on the importance of early experience to children’s brain 
development and later cognitive functioning highlights the importance of the early years. Child 
development literature and evaluation findings from programs that started during infancy 
confirm the fact that such early intervention can produce positive effects for both parents and 
children. 

Evidence from Program Experience — Most of the programs examined that had an 
influence on child development or parenting started intervention at birth or in early infancy. Of 
the 18 programs that displayed effects on parents and children, 16 started during the first year of 
life. Of the 20 programs that showed child effects, 17 started prenatally or during the first year 
of life. All of the programs we examined enrolled children before they were three-years-old. 

The programs that started prenatally and continued during early childhood (4 of 20 programs) 
affected child health in addition to other child development areas. These powerful findings 
support the belief in both the family support and child development fields that providing an early 
start is important. 

While intervening early is important, it is also important to acknowledge that later 
experiences also have significant implications for parents and children 13 The tasks of parenting 
and the skills upon which a child relies change as the child develops 14 and the outcomes of 
concern are different. While parents may have little difficulty with parenting during their child’s 
infancy, they may experience more problems when faced with the challenging toddler and the 
autonomous preschooler. Risks and stresses change over time. A family could be at low-risk at 
birth but at higher risk at 6 or 12 months. If the quality of parenting changes radically during 
later years, the effects of early good parenting may fade. Children’s quality of relationships 15 
and the course of their development can change after infancy. Compromised parenting that 
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occurs later in childhood can lead to poor parent-child relationships that may have corresponding 
poor effects for children. 

The developmental literature on this topic suggests that the availability of services or 
supports into later childhood (beyond age 4) is important. Continuity can take a variety of forms, 
such as appropriate services in later childhood, or coordination with other programs such as 
Head Start, after school programs, or parent involvement programs when the child enters school. 
It can also mean developing the capacity of parents to maintain a good relationship with their 
children and ensuring parents have access to supports and resources when needed. 

4. Have reasonable expectations about the time commitment required of families and 
providers. 

Intensive intervention, defined in terms of actual time and frequency of contact, has long 
been advocated by researchers 16 . The general perception has been that to improve child 
development, frequent and lengthier interventions produce more pronounced or long-lasting 
outcomes. While intensive programming can lead to strong outcomes, high intensity services 
may not be consistent with the practical time and resource constraints that programs and families 
now face. Families might not want intensive services or may be receiving more service time 
from different agencies than they can manage. As a result of the welfare reform legislation 
recently passed by Congress, many families have a limited amount of time during the day to 
spend in family support programs Employment stipulations for welfare recipients require many 
low-income parents to spend more time at work and less time at home or in programs with their 
children. Backing off on the provision of services as a result of this would be unfortunate. 
Programs may have to get to parents and children in different ways such as providing home visits 
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at alternative times of the day or week and providing individualized services to children when 
they are with other caregivers. A promising approach is to have mothers volunteer in their 
children’s child care or preschool classroom as a way to satisfy one of the welfare 
requirements 17 . In addition, the need to cut costs and serve families on a restricted budget also 
requires that providers have only a limited time to spend in direct service to families. As a result, 
programs should carefully develop the frequency and quality of time families and providers 
spend in direct service contact, without sacrificing service quality. More frequent contact for a 
shorter time (shorter visits more frequently) can be more effective for and more easily accepted 
by some families. Families should not be cut off from services because of their more stringent 
schedules. 

Balancing time and resource commitments can mean, for example, that programs develop 
the capacity of participants to develop networks of support in their communities that can be used 
when providers are not available or families do not have large chunks of time to devote to direct 
services. This must be supported in its development and implementation. It may also mean that 
programs focus on sustaining the ability of families to continue to be good parents and obtaining 
support when direct services end. Alternative services such as hotlines and services that are tied 
into Head Start or other child care arrangements or the church-based programs may be necessary. 

Evidence from Program Experience — Most of the 30 programs examined required a time 
commitment of less than or equal to two hours per week. Most, however, were able to show 
positive impacts on both child and parent outcomes. For example, the Mother-Child Home 
Program model required frequent 30-minute home visits, but had effects that lasted well beyond 
the end of program participation (see also the Pittsfield Parent-Child Home Project profile). 
Programs must carefully balance the needs of providers and families with the time and resources 
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available to each and pay careful attention to information from past experience about what 
works. 

5. Recognize that factors in parents’ social and cultural context have an impact on 
parenting. 

Most family support programs are grounded in an ecological theory of human 
development which says that the parent-child relationship is one variable among many that affect 
the parent and child’s development 18 . This theory of development acknowledges the influences, 
both direct and indirect, of the neighborhood, the community, and the entire society on the 
developing child. For example, neighborhoods influence children in a variety of ways 19 . A 
parent may display all of the behaviors that would lead to positive child outcomes in a safe and 
nurturing environment, but, if the parent is raising a child in a neighborhood lacking such 
qualities, the effects of these same parenting practices may not be as strong 20 . In addition, certain 
community environments may elicit parenting behaviors that do not allow children to thrive 
developmentally 2 '. For example, a mother who is concerned about her child’s safety may keep 
the child in the home and restrict the child’s peer contact. While this type of restriction is more 
likely to keep the child alive, it may also deprive the child of experiences that would lead to 
maximum adaptation to his or her environment. 

Poverty and other facets of socioeconomic status also influence parenting and child 
development. Because of the psychological distress associated with living in poverty, mothers 
who are poor tend to use more power-assertive techniques when disciplining their children and 
are, overall, less supportive of their children than are more advantaged parents. Economic 
hardship also is associated with less expression of affection and less visible responsiveness from 
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parents. 



Evidence from Program Experience — While programs acknowledge these influences, 
most are not equipped to directly address these issues 22 . The services provided by most of the 
programs examined focus on parents and children with the predominant goal of improving 
parenting as a way to maximize child development. While programs can be effective in 
improving parenting, these effects may wear off quickly in an environment that is not supportive 
of such effects. Although it may be outside of their direct purview to address each of these 
variables that influence parents and children, programs need to use tactics such as collaboration 
with other service systems to mitigate the negative effects of these problems on parenting. 

The realization that multiple variables affect parents and children is reflected in a 
movement toward more community-based and collaborative family support initiatives. Such 
programs focus on improving parenting while they attend to other family needs. For example, 
these initiatives often focus on strengthening informal supports among community members, or 
act as advocates for the improvement of community job and economic opportunities. These 
programs fill gaps in services; “the goals, emphases, and types of services provided. . . are shaped 
by local conditions and concerns, and by strengths and weaknesses in other local helping 
services .” 23 The move toward community-based initiatives is thus “part of a trend to provide 
more comprehensive services for children and families... and to integrate more fully housing and 
economic development with family support services in neighborhoods and communities .” 24 

Family support and parenting programs need to consider where their services fit within 
the overall system of services in their communities and to consider how they can work with these 

25 

services to meet broad family needs. 
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6. Work with other providers to form a system of efficient and comprehensive services. 

Researchers suggest that family support and parenting programs with a broader and more 
comprehensive range of services have stronger effects on children 26 because the focus of such 
programs is on helping participants attain their full potential in all areas of the child’s and 
family’s development. They believe that “sustained and comprehensive early intervention can 
produce substantial effects.” 27 

Families often need access to a variety of services to sustain positive parenting practices. 
While family support and parenting programs alone cannot provide each of these services, they 
can work with other providers to ensure that families have access to a broad range of services. 
They can help families access other available services through referrals. In addition, by putting 
parents in touch with one another, programs can foster the development of a network of supports 
among parents. 

Evidence from Program Experience — Many of the 30 programs examined that produced 
positive outcomes, provided participants with a variety of services, and served children and 
parents separately as well as together. Most of the programs (22 of 30) with positive effects had 
between three and five services available to families (two to four basic services such as home 
visits, structured parent education, and one to three additional services such as referral to other 
services) versus just providing one or two discrete services. In addition, some programs served 
families both in and outside of their homes, and provided referrals to services for families with 
needs beyond those met by their direct services. 

7. Use evaluation strategies that reflect the field’s move toward more comprehensive 
initiatives. 

As the field of family support moves toward the implementation of more complex and 
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comprehensive initiatives, programs need access to better information about the success of such 
strategies. This may mean collaborating with other local service providers to track families 
across services or developing the capacity within programs to track information that is useful to 
improving their own services and outcomes. It may also mean that many programs are ready to 
move beyond a traditional reliance on experimental and quasi-experimental designs common to 
research and development efforts toward more cost-effective and alternative forms of evaluation. 

Although respected in the research and evaluation arena, experimental and quasi- 
experimental designs have a somewhat limited applicability for the evaluation of comprehensive 
programs. The variation in services across families makes it difficult to examine the direct link 
between services and outcomes using these traditional designs. In addition, recruiting an 
appropriate comparison group for such studies is difficult in a community that offers 

comprehensive services. Most families are likely to be either directly or indirectly influenced by 

28 

one or many of the services available 

Evidence from Program Experience — Experimental and quasi-experimental evaluation 
designs, although powerful in their ability to attribute program effects to the intervention, may 
not be cost-effective ways of measuring the effects of many programs. Many of the programs 
examined were highly-funded demonstration programs with a large amount of resources for 
evaluation. Smaller community-based programs may not be able to generate the level of funds 
often required for these evaluation designs. These programs are, however, equally accountable 
for measuring program outcomes. 

The search continues for a good model of evaluation for more comprehensive and 
complex programs. This recommendation encourages programs of this type to adopt any of 
several promising evaluation strategies. For example, process and implementation studies, along 
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with other quantitative and qualitative methods, can provide valuable process and outcome data. 
In addition, participatory evaluation is a promising approach that involves all people involved 
with a program in the evaluation effort 29 . All involved in the program are involved in the 
evaluation. The view is that people involved in a program know the most about the program. 
Their expertise is seen as invaluable to the evaluation process and is integral to participatory 
evaluation. This strategy focuses on strengthening the ability of program managers and 
administrators to conduct their own evaluation in the future. The goal is that the information 
generated will be better utilized by increasing the involvement of its potential users. This 
approach leads programs to become self-evaluating and learning organizations, engaging in a 

process that leads to continuous improvement and an improved ability to meet the demands of 

* 

accountability. It is important that program staffs ability to conduct such evaluation be taken 
into account when designing an evaluation strategy. While some critics raise issues about 
programs evaluating themselves, the potentially greater use of such evaluation findings 
outweighs this pitfall. 

8. Examine child and parent outcomes and needs longitudinally. 

Although programs may serve children and parents for only a specific period of time, it is 
important that they build the capacity to continue measuring program results over time. While 
the obvious benefit of such examination lies in its potential ability to capture the program’s long- 
term success, it is equally beneficial to use such an opportunity to periodically assess the long- 
term needs of parents and children. In this way, data collection over time is not only beneficial 
for purposes of evaluation, but is beneficial for those who are served. 

While acknowledging the need for long-term evaluation, it is important not to ignore the 
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desire on the part of decision makers for short-term evidence. Short-term indicators of success 
should also be incorporated into any evaluation. 

Evidence from Program Experience — The evidence from our examination in favor of 
longitudinal evaluations is compelling. Several of the programs studied participating families 
and children for many years after program participation ended. These programs provide 
evidence that some child effects last through adolescence and teen years, and some effects may 
not become apparent until much later in a child’s life. 

For example, the Pittsfield Parent-Child Home Program, a replication of the Mother- 
Child Home Program, enrolled children from ages two to four years for semiweekly, one-half 
hour home visits. The evaluation followed children through the end of high school. Children 
who completed the program dropped out of school at significantly lower rates (15.7% vs. 40%), 
and had higher rates of high school graduation (84.1% vs. 46.2%), than children in the same 
school district who did not participate in the program or other services offered in the community. 

The Yale Child Welfare Research Program conducted a follow-up evaluation of its 
participants 10 years after program completion. Children in the intervention group had better 
school adjustment, attendance, and behavior than comparison group children from the same 
neighborhoods. In addition, mothers completed more years of school, were more likely to be 
married, were more often self-supporting, and were more likely to initiate involvement in their 
children’s schooling. 

Some of the most striking long-term effects were observed in the evaluation of the High 
Scope/Perry Preschool Project (not summarized here). Children who participated in this 
program as preschoolers were more likely to be self-supporting, gainfully-employed adults 
nearly 22 years after the program ended. 
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Most of the programs that tracked outcomes longitudinally examined only child 
outcomes over time. Presumably, many either stopped tracking parents because the child’s 
development was of primary interest, or a lack of resources did not permit such a long-term 
examination. In other cases, evaluations that tracked parents continued to measure the same 
parent outcomes over time with little attention to developmental changes that occur naturally 
during the parenting process. As children progress developmental^ and confront new 
developmental tasks, the important features of parenting change as well As stated previously, it 
is important that programs prepare parents for these changes, or provide a way for parents to 
continuously learn as their children develop. If programs provide for this continuity of effects, it 
is important also to measure them. Given the changing tasks of parenting, it is important to track 
parents’ development in addition to children’s. Good measures of parent development must be 
developed if this is to be accomplished. 

Because of a paucity of funds, programs almost always are forced to curtail their 
evaluations soon after the program ends. Programs that develop the capacity to follow parents 
and children, however, can use this process to better meet the demands of accountability, as they 
can potentially make the case for results that endure over time. In addition, tracking parents and 
children can allow programs to assess the changing needs of parents and children, and improve 
their services to develop the capacity of parents to successfully meet future challenges. 

9. Choose measures that reflect intended program outcomes. 

As the need to measure outcomes intensifies for programs, they must be confident that 
their measures adequately reflect their progress in achieving intended results. It is important that 
all involved are clear about program goals and the related, measurable objectives before they 
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attempt to choose measures. This means that programs should carefully choose instruments or 
indicators that will be meaningful 30 . This also means reassessing the usefulness of traditional 
measures and using measures or indicators that are equally reliable and useful but, perhaps, more 
efficient to use. Many of the results of interest to parenting programs do not have good measures 
available Programs cannot be expected to show that they affect results if they cannot adequately 
measure the relevant results. 

Programs may at times, exhibit an overreliance on standardized measures that are not 
appropriate to the goals, services, and intended outcomes of the program. Rather than relying on 
such instruments, or frequently-used traditional measures, programs and evaluators should 
choose measures that explicitly reflect the results of their services and push for the development 
of such measures if they are not available. 

Evidence from Program Experience — The variety of programs examined used many of 
the same outcome measures. While several of these measures are well-respected in both the 
child development and evaluation domains, programs must be careful not to adopt measures 
based only on their reputation. Equally important to reliability and validity concerns is the 
concern that the data collected from the measures can be interpreted and used in meaningful 
ways. If a measure does not seem adequate, programs should do all they can to push for better 
measures. 

At least half of the programs examined used the Home Observation for Measurement of 
the Environment (HOME) to measure parenting 31 . While a popular instrument for assessment 
among both program evaluators and child development researchers, questions exist about its 
usefulness for measuring parenting outcomes. For example, the HOME is a broad inventory of 
the home environment and contains only a few items that are interactional in nature. In addition. 
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few of the interaction items on the inventory are likely to be observed by the home visitor 
conducting the assessment 32 . Concerns also exist that participation in a parenting intervention 
may allow parents to guess what the HOME measures and thus lead to biased conclusions about 
the success of the intervention 33 . 

If a program desires to measure parenting as link to child development, evaluators should 
choose parenting measures that assess the parent-child relationship. The aspects of parenting 
identified in the first recommendation as important areas for intervention (i.e., sensitivity, 
responsiveness, emotional availability) are also worthy targets for measurement Researchers 
have developed instruments that are equally efficient to administer as the HOME but which may 
be even more useful in their measurement of parenting. Observational measures of sensitivity, 
and observations of mothers 34 in problem solving situations are examples of such techniques. 
Interactional measures can be difficult to administer and may require staff training but are worth 
examination because of the specific way they relate to program goals. 

The programs we examined also relied heavily on intelligence quotient (IQ) as a primary 
child cognitive outcome indicator. Twenty-two of the 27 programs that measured child 
outcomes used some standard measure of intelligence or developmental quotient as an indicator 
of child cognitive abilities. The danger of using IQ as the primary indicator of cognitive 
development is apparent in the initial evaluations of Head Start . IQ was used as a program 
outcome and led to conclusions that the program was not effective because it failed to produce 
significant, lasting changes in IQ. 

Even when significant differences are achieved, it is hard to make real world conclusions 
from IQ results. While a difference in average IQ of 10 points may be statistically significant, 
the clinical significance of a 10-point difference in IQ is minimal in terms of an individual’s 
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chance of success over another individual’s. In addition, IQ is not a strong predictor of success 
in school and later life. Traditional measures of intelligence do not take into account other 
cognitive abilities that affect achievement and success later in life, and do not address the issues 
of what constitutes an appropriate definition of intelligence. Traditional IQ tests focus on verbal, 
spatial, and other “performance” abilities. A growing body of knowledge has led to an 
expansion of the definition of intelligence beyond these traditional realms. Other cognitive 
indicators such as language and problem solving skills may be more relevant to later success in 
school than is IQ. 

The examples of the HOME and the IQ are used to illustrate the need to carefully 
consider the measures used to capture program outcomes. This recommendation does not 
advocate their total abandonment for evaluation purposes, but rather encourages providers and 
evaluators to contemplate the implications of their measurement choices. It also highlights a 
need for the development and dissemination of a greater number of choices of measures. 

10. Examine the relation between parent and child outcomes. 

Programs that carefully track child and program outcomes should include provisions for 
looking at changes in parent outcomes in relation to changes in child outcomes. By showing 
statistical relations between positive effects on parent and child outcomes, programs can make 
stronger and more valid claims regarding their effectiveness. Such claims have a direct 
consequence for programs that are accountable for proving their results. 

Most programs attempt to improve parenting as a way of maximizing child development 
and, therefore, measure both parent and child outcomes. If the evaluation results show positive 
differences for both sets of outcomes at the end of the intervention, many programs claim to have 



O 

ERIC 



Harvard Family Research Project 



27 



25 



attained their goal of influencing child development by improving parenting. Most, however, do 
not establish this link statistically. Evaluations that do not empirically link the two domains are 
justified only in making separate claims about their influence on parent and child outcomes. 

Evidence from Program Experience — Evaluators and providers can justify more valuable 
conclusions by using strong, but simple, analyses of outcome data. Only a handful (5) of the 30 
programs conducted such analyses. For example, the United Charities of Chicago Family 
Development program found positive effects on maternal teaching style and child cognitive 
development and play when children were two-years-old. The evaluation examined the links 
between parent and child outcomes, and found that certain aspects of maternal behavior were 
associated with child cognitive scores. This finding allowed the researcher to draw a stronger 
conclusion about the link between parent and child outcomes. 

In addition, the Houston Parent-Child Development Center, the Haitian Perinatal 
Intervention Project, and the Parents as Teachers (PAT) National City evaluations tested the link 
between HOME scores and child outcomes using multiple regression techniques which allow 
one to use one outcome areas to explain changes in another. This technique allows researchers 
to start talking about the causes of changes. In the PAT evaluation, the HOME score predicted 
all of the measured child development outcomes. For the Houston PCDC and Haitian PIP 
evaluations, the HOME predicted child cognitive outcomes. The Mother-Child Home Program 
evaluation found that maternal verbal interaction was related to child behavior traits, HOME 
scores, and child IQ. 

While this recommendation is not difficult or time-consuming to carry out, it is valuable 
for programs facing the increasing demands of accountability. Simple analyses can add 
increased credibility to the basic tenets that underlie the theories of family support and parenting 
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programs. 



11. Establish mutually beneficial relationships between evaluators, providers, and child 
development researchers. 

The gradual expansion of family support and parenting programs beyond research and 
development efforts alluded to in earlier recommendations has also meant a gradual shift toward 
alternative forms of evaluation that require evaluators to work closely with providers in 
mutually-beneficial relationships. Such evaluation strategies are suited to the efficiency and 
measurement demands of accountability. This recommendation calls for a continuation of such 
collaboration, as well as an expansion of this practice. It encourages evaluators to help programs 
develop the capacity for self-evaluation and to assist programs in translating outcome 
information into useable formats for policymakers. 

Collaboration can also occur between program evaluators and child development 
researchers to better inform the field of child development. Programs that focus on maximizing 
child development through better parenting, constitute an ideal proving ground for child 
development theory 36 . Several of the programs and evaluations examined were developed by 
child development researchers, and explicitly reflect the theories of the field. These connections 
between the two fields allow evaluators to test the relations between variables of parenting and 
child development that are proposed in theory. 

A promising area for such a collaborative dialogue is in the examination of the ecological 
model of child development (see recommendation #5). This theory is implicit in the theories of 
family support programs and deserves close examination. An evaluation of a program could 
examine various contextual variables that affect families. This could be a wonderful chance to 
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more thoroughly test the assumptions of the ecological model. 

It is also important that both child development researchers and evaluators act to improve 
evaluation. Evaluators can examine links between parenting and child development that are well 
documented in child development research. Evaluators can also incorporate measures of the 
aspects of parenting that developmental research suggests have the most influence on child 
development and help to develop measures if they don’t exist. In addition, a large body of 
evidence addresses how parents influence their children’s development. By examining links that 
are already well established, evaluators can make more definite conclusions about the 
effectiveness of a given program. 

These types of mutually-beneficial relationships assist each of the groups involved in 
meeting the demands of accountability. By collaborating to improve and build the capacity for 
more valuable evaluation, programs can meet the demands for results with a higher level of 
confidence in the interpretation of their findings. 

12. Consider using cost-effectiveness analyses as a method for measuring and reporting 
program results. 

Cost-effective analyses determine the effectiveness of a program in achieving results in 
relation to how much the program costs. They identify and compare the cost of the program and 
actual or anticipated costs of other alternatives and determine how much it costs to achieve a 
desired change. In addition, they provide information that aid comparisons about the relative 
utility of interventions . 37 While the procedures used to conduct these analyses are often 
technical and complex and require training, they provide valuable decision making criteria for 
policymakers and providers. Because of their value, cost-effectiveness studies should be 
considered as a possible part of program evaluations . 
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Few programs conduct cost-effectiveness analyses. In many cases, they lack the 
resources or methodological expertise needed to perform such studies. If policymakers and 
funders expect that cost-effectiveness information be provided to them, they must provide the 
necessary supports and resources. In other cases, it is difficult to attach monetary values to 
program outcomes and forecast savings for the future. Programs that conducted cost studies, 
however, have produced estimates that are invaluable to decision making processes about family 
support programs across the nation. 

Evidence from Program Experience — The Prenatal/Early Infancy Project found that the 
program cost $3,246 per family per year (in 1980 dollars) for families overall, and $3,133 per 
low-income family per year (includes costs for ancillary services). Savings for the government 
were estimated by comparing treatment and control groups on expenditures for four government 
programs (Aid to Families with Dependent Children, Food Stamps, Medicaid, and Child 
Protective Services), as well as tax revenue from mothers’ earnings. Findings estimate cost 
savings at $1,772 per family for all families involved, and $3,498 per low-income family in the 
program. 39 Results thus show long-term government savings for investment in home visiting 
programs like PEIP. 

The estimates present results in ways that are both understandable and usable. They 
translate the effects of programs into the “bottom line” (i.e. dollar benefits or their equivalent), 
and supply criteria for decision makers who have to weigh program costs against their future 
value. 

Programs face a challenge if they are to build their capacity to conduct cost-effectiveness 
analyses. Guides that outline the procedures for conducting such studies are now available, 
however, increasing the capacity of family support and parenting programs to conduct cost 
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analyses in the future. 40 The greatest challenge lies in finding adequate resources for their 
implementation and collecting the types of data needed to translate program effects into dollar 
figures. 

Implications of the Recommendations 

These 12 recommendations lay the groundwork for future decisions about family support 
and parenting programs that are facing demands for demonstrable results and increased 
accountability. This paper takes valuable information from the rich history of large-scale 
program and evaluation efforts and summarizes what has been learned about program services, 
implementation, and evaluation. It is crucial that a conversation about the points raised here 
occur among policymakers, program designers and providers, and evaluators. If the issues raised 
in the recommendations are not incorporated into this conversation, there is a risk that desired 
results will not be achieved or will not be demonstrable even when they are achieved. 

Program evaluation is crucial in this era of accountability. Policymakers allocating 
money for a new or existing program must provide enough money for thorough evaluations. 
Providers and evaluators must work together to make sure that the evaluation accurately captures 
the outcomes of program elements and to ensure that evaluation results are useful to all 
stakeholders. 

While they are ambitious, the recommendations are grounded in an acknowledgment of 
the changes in program resource allocation and policy migration toward programs that address 
the need for collaboration, integration of services, and increased accountability. Overall, they can 
contribute to thinking about: 



ERIC 



Harvard Family Research Project 



32 



30 



• Improving parenting program design and service provision. 

Program design and service provision must reflect the lessons learned through evaluation, 
research, and experience. The recommendations lay out several ingredients that are important 
for the success of programs intending to improve parenting. Such interventions must start 
early and provide services that focus on and seek to modify parent-child interactions directly. 
They must be aware of and address factors that can affect parents’ investment programs such 
as regional, cultural, and ethnic beliefs, values, and practices. They need to recognize the 
factors in parents’ social and cultural context that have an impact on parenting. Services 
must be scheduled with a realistic understanding of the time constraints of families and 
providers. It is crucial that a system of efficient and comprehensive services be put in place. 

• Improving the evaluation of parenting programs. 

Evaluations must include better outcome measurement and implementation assessment. 
Measures are needed that more accurately reflect intended program outcomes than do many 
currently in use. The common program goal of improving child development through 
improvement of parenting must be examined by looking at the relation between parent and 
child outcomes. Evaluation time frames need to be long enough for hoped for program 
outcomes to be affected. In this era of accountability in which program effectiveness may be 
tied to funding for program continuation, cost-effectiveness studies are a promising 
approach. 

• Adding to the utilization of evaluation results. 

Results should be used for organizational change and learning. To ensure that this happens, 
relationships between evaluators, providers, and researchers in relevant fields must be 
fostered. Evaluation needs to be an ongoing and collaborative effort. To be truly useful to 
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all stakeholders, evaluations must reflect and support the family support field’s move toward 
more comprehensive initiatives. 



Here, concrete criteria derived from a synthesis of critical information are set forth. These 
are necessary for the planning of the next generation of programs focusing on improving 
parenting. The hope is that the recommendations can fulfill their potential by expanding the 
contribution of evaluation research to decision making. This work starts a movement from the 
research and development era in which programs researched, demonstrated, and died into one 
that makes better use of lessons learned from the intense labor put into these programs. 
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APPENDIX A 



Family Support Programs Examined 

AVANCE Parent Child Education Program 
CEDEN Parent-Child Program, TX 

Child and Family Resource Project: Infant-Toddler Component 
Child Development Center, Birmingham, Houston, New Orleans 
Dad’s Day, MD 

Gordon Parent Education Infant & Toddler Program, FL 

Haitian Perinatal Intervention Project, FL 

Hospital and Home Support During Infancy 

Infant Health and Development Program 

Information and Insights about Infants, NY 

Mailman I and II: Home-Base Intervention for Teen Mothers, FL 

Maternal Infant Health Outreach Worker Program, KY, TN, VA, WV 

Missouri Parents as Teachers, MO 

Mother-Child Home Program, NY, Bermuda, MA 

New Parents as Teachers, MO 

Parent Infant Project, CO 

Parents as Teachers, CA, National City, TX 

Prenatal Early Infancy Project 

Project Giant Step 

Project Redirection Demonstration 

Public Health Home Intervention 

Rural Alabama Pregnancy and Infant Health Program, AL 
Syracuse University Family Development Research Program, NY 
United Charities of Chicago Family Development Program, EL 
Yale Child Welfare Research Program, CT 
Ypsilanti-Carnegie Infant Education Project, MI 
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29 This is a potentially fruitful area to focus on in the implementation of recommendation #11. 
Child development research has tried to capture some of these more difficult areas. 
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31 Howrigan, 1988 

32 Howrigan, 1988 

33 Jacobs, 1988; Westinghouse Learning Corporation/Ohio University, 1969 

34 Weiss & Halpern, 1991 

35 Rossi & Freeman, 1993 
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their results when these cannot all be reduced to a single dimension of payoff (typically money); 
Scriven, 1991. 

37 Cost savings are conservative because they do not include potential future savings beyond 
children’s age at four years. 
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